Discussion of Boosting Papers
نویسندگان
چکیده
We congratulate the authors for their interesting papers on boosting and related topics. Jiang deals with the asymptotic consistency of Adaboost. Lugosi and Vayatis study the convex optimization of loss functions associated with boosting. Zhang studies the loss functions themselves. Their results imply that boosting-like methods can reasonably be expected to converge to Bayes classifiers under sufficient regularity conditions (such as the requirement that trees with at least p+ 1 terminal nodes are used, where p is the number of variables in the model). An interesting feature of their results is that whenever data-based optimization is performed, some form of regularization is needed in order to attain consistency. In the case of AdaBoost this is achieved by stopping the boosting procedure early, whereas in the case of convex loss optimization, it is achieved by constraining the L1 norm of the coefficient vector. These results re-iterate, from this new perspective, the critical importance of regularization for building useful prediction models in high-dimensional space. This is also the theme of the remainder of our discussion. Since the publication of the AdaBoost procedure by Freund and Schapire in 1996, there has been a flurry of papers seeking to answer the question: why does boosting work? Since AdaBoost has been generalized in different ways by different authors, the question might be better posed as; what are the aspects of boosting that are the key to its good performance?
منابع مشابه
Discussion of "the evolution of boosting algorithms" and "extending statistical boosting".
This article is part of a For-Discussion-Section of Methods of Information in Medicine about the papers "The Evolution of Boosting Algorithms - From Machine Learning to Statistical Modelling" and "Extending Statistical Boosting - An Overview of Recent Methodological Developments", written by Andreas Mayr and co-authors. It is introduced by an editorial. This article contains the combined commen...
متن کاملBoosting - an unusual yet attractive optimiser.
This editorial is part of a For-Discussion-Section of Methods of Information in Medicine about the papers "The Evolution of Boosting Algorithms - From Machine Learning to Statistical Modelling" and "Extending Statistical Boosting - An Overview of Recent Methodological Developments", written by Andreas Mayr and co-authors. It preludes two discussed reviews on developments and applications of boo...
متن کاملA Comparative Analysis of Metadiscourse Markers in the Result and Discussion Sections of Literature and Engineering Research Papers
This study compares metadiscourse markers in result and discussion sections of literature and engineering research papers. To this end, 40 research articles (20 literature and 20 engineering) are selected from two major international journals. Based on Hyland’s (2005) model of metadiscourse, the articles are codified in terms of frequency, percentage, and density of interactive and interactiona...
متن کاملSome Theoretical Aspects of Boosting in the Presence of Noisy Data
This is a survey of some theoretical results on boosting obtained from an analogous treatment of some regression and classi cation boosting algorithms. Some related papers include [J99] and [J00a,b,c,d], which is a set of (mutually overlapping) papers concerning the assumption of weak hypotheses, behavior of generalization error in the large time limit and during the process of boosting, compar...
متن کامل